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Abstrac t 



Language testing has a large number of commercial applications in both the institutional 
and the private sectors. Some jobs in the health services sector or the public services 
sector require foreign language skills and these skills require continuous and efficient 
language assessments. Based on an experience developed through the cooperation of 
Antwerp University and the Polytechnic University of Valencia, a language testing 
platfonn was used to produce simple and multi-language skills assessments for a 
publishing company. Results indicate that academic low stakes testing platforms can 
replace efficiently the use of larger tests or costly language testing companies that either 
are expensive, or just do not tailor the tests according to their customer specific needs 
properly. Since academic institutions work on a non-profit basis, the revenues can be 
used for further research or improving their current developments. This paper is 
especially important because it seeks to show very uncommon use of educational 
software and platforms in productive commercial ways. 
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Abstract 



Language testing has a large number of commercial applications in both the institutional 
and the private sectors. Some jobs in the health services sector or the public services 
sector require foreign language skills and these skills require continuous and efficient 
language assessments. Based on an experience developed through the cooperation of 
Antwerp University and the Polytechnic University of Valencia, a language testing 
platfonn was used to produce simple and multi-language skills assessments for a 
publishing company. Results indicate that academic low stakes testing platforms can 
replace efficiently the use of larger tests or costly language testing companies that either 
are expensive, or just do not tailor the tests according to their customer specific needs 
properly. Since academic institutions work on a non-profit basis, the revenues can be 
used for further research or improving their current developments. 
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Background 



A large number of private companies demand a second (or subsequent) language to 
undertake certain jobs in a country or abroad: nurses living and working abroad, for 
example international telephone operators, international executives or business people 
(Gross, 2004; Clarke, 1999), and health professionals (Moreno et al., 2007). Thus, 



language assessments are absolutely necessary for many private companies (Chahoulb 
& Tarone, 1996). This demand has been growing over the last 10 years due to the 
mobility of working labor as well as the increasing demand for certain jobs in specified 
European and American countries (Maiworm & Teichler, 1996), such as those related to 
health. Private companies' capacity to communicate effectively with their workers is 
also necessary and crucial to the company's organization and quality service delivery. 
This is especially true when potential clients are culturally and linguistically diverse. 
Therefore, private companies rely on costly and not always efficient tests to assess their 
workers. Overall, the most commonly accepted tests worldwide are IELTS, TOEFL, 
BULATS (The Business Language Testing Service), and OET (Occupational English 
Test) (O’Neill, 2007; Iwashita & Gove, 2003). However, these tests may not be able to 
cope with the companies' or institutions’ needs especially in the health sector (McCabe 
et al., 2008). They also usually involve a more or less lengthy process that includes 
registration, test delivery, marking, and return of the results. The process is usually 
expensive, inefficient and by the time the results are returned one (or some) prospective 
workers may have decided to look for another job; as a result, the company loses its 
money and part of the selection process is a waste of time. 

On the other hand, academic centers worldwide have the resources to develop especially 
tailored courses and assessments for their own students. This is not just for languages 
but also for general core subjects. Generally, these assessments are low-stakes tests and 
permit assessments to be made within a few days, from the time they are programmed 
to the moment when the results are returned (Rouse & Krueger, 2004). In general, these 
low stakes tests require different security systems and, overall, are tailored to each 
specific case. Additionally, their use is not homogenous all through the year. There are 
usually peaks and periods with limited use; for instance, in the European context, May- 
June may be a peak, but February-March is a period with minimal use. 

The Polytechnic University of Valencia has developed an oral- written flexible testing 
system for oral evaluation that has been described in some academic papers and is 
currently developing a second version. On recent trials, it was seen that although the 
system was allowed for the whole 25,000 student population under demand, only a few 
professors requested its use. Thus the question was, why not offer it to a private 
company? As a result, the testing platform was offered to a publishing company twice a 
month for a free-of-charge teacher training course on the prospective University 
Entrance Examination language test (details below). 25 teachers used the academic 
platfonn. 



Goals 



The main goal of this case study was to see whether the platform could cope with the 
different needs required of it, and whether it could function as an efficient and low cost 
tool for taking tests by these teachers. The secondary objectives were as follows: 





1 . Analyzing the platfonn’s flexibility and adaptability to specific 
demands (suggested by the private company), 

2. Suggesting a pattern of use for private companies, 

3. Addressing guidelines of good practices in using academic software 
for commercial purposes, and 

4. Suggesting ways to implement academic language testing platforms 
for commercial use. 



Subjects 



The teacher trainees took a number of trial tests in order to observe and develop a 
special sensibility towards computer-assisted language testing. The only incentive that 
they received was a course certificate and a deeper understanding of computer-based 
language testing processes. However, they always knew that the University Entrance 
Examination would differ to the mock tests they were taking, in length and difficulty. 
However, since the main goal of this paper is related to the flexible use of language 
testing platforms, we will no go any further into the subjects' specifics. 



Description of the activity 



During the academic course of 2007-2008, the Universidad Politecnica de Valencia 
(UPV) finished a prototype of an online test for foreign languages that included both 
oral and written sections. The platfonn was designed to evaluate: During the academic 
course of 2007-2008, the Universidad Politecnica de Valencia (UPV) finished a 
prototype of an online test for foreign languages that included both oral and written 
sections. The platform was designed to evaluate: During the academic course of 2007- 
2008, the Universidad Politecnica de Valencia (UPV) finished a prototype of an online 
test for foreign languages that included both oral and written sections. The platfonn was 
designed to evaluate: During the academic course of 2007-2008, the Universidad 
Politecnica de Valencia (UPV) finished a prototype of an online test for foreign 
languages that included both oral and written sections. The platfonn was designed to 
evaluate: 



1 . Multiple choice and objective responses, 

2. Listening tasks prompted either by listening or listening and watching 
prompts and input, 

3. Writing tasks either by listening or listening and watching prompts 
and input, 



4. Reading sections, and 





5. Speaking, either by listening, or listening and watching prompts and 
input. 

Since the use given to that platform until that moment had been merely for research 
purposes, it was thought that some professional use could be added to regular use in the 
academic environment. At that time, Heinemann Iberia was just about to begin a teacher 
training course in language testing. So the publishing company suggested using the 
platform and the UPV considered the possibility to use it for testing for commercial 
purposes. Since the relationship between the Universidad Politecnica de Valencia and 
Antwerp is a long standing one- first through the Didascalia Institute and today through 
Linguapolis- , the UPV requested advice from them. As a consequence, the Belgian 
institution recommended delivering the tests as follows: 



Objective 


Initial assessment: 
Teachers would 
experience the web 
based tool 


Individual skills, 
grammar and vocabulary 
assessment 


Final assessment 


Computer 
platform use 


Full exam was 
delivered 


Separate types of exam 
were delivered: 

Multiple choice; Written- 
listening; written- 
reading; Reading- 
Listening; and so on 


Full exam was 
delivered (four 
skills plus multiple 
choice) 


Approximate 

dates 


January 2008 


February-April 2008 


June 2008 



Teachers took either one-skill tests as well as integrated tasks tests. Results for the 15 
teachers were delivered in two days because the raters were volunteers. However, these 
tests (either written or oral) could be rated almost on the same day, if professional raters 
were used. Additionally, multiple choice tasks were self-corrected. The publishing 
company provided the contents for the test so the takers had to take both general and 
very specific tests. 



Customer satisfaction 



Being the academic testing platform, we expected the teachers to feel at ease from the 
beginning. Thus, after the first session a twelve-item questionnaire was distributed. 
Results indicate that teachers were rather satisfied with use given to the platfonn as a 
whole. The teachers stated that the tutorial was one of the major achievements. As a 
consequence, it is important that if potential customers are not familiarized with the 
platform, at least to include a clear and user friendly tutorial, and if the computer 
platform is not as straightforward as possible in its use, adaptive to its purpose and 






goals, the client may not understand that the platfonn is currently measuring what is 
intended to (face validity). 



Evaluative comments and conclusions 



The activity was considered positively. Teachers valued saving time in their 
assessments and also were happy because each individual activity and task 
corresponded exactly to what was necessary and expected in each moment and course 
stage. The older teachers acknowledged the difficulty of taking online tests when their 
computer skills were limited but also recognized that handling the computer tool had 
been simpler than they thought at the beginning. Eventually, most of them got used to 
the computer tool and believed they perfonned almost free of anxiety and computer 
presence awareness. Most of them also believed that taking the test would be 
appropriate in other professions and professional situations. 

Teachers or prospective test takers would have to face different activities according to 
the hiring company. The great advance as opposed to the standardized tests is that, like 
in the case hereby described, the company states the conditions and suggests the kind of 
materials to be included in the test. Furthermore, automatic rating devices (like e- 
rater™ or a speech recognition system may be designed to a considerably lower cost 
than only 6 years ago). Thus, tailoring language tests may be faster, less costly and 
more efficient that hiring large companies like ETS or the Cambridge Board of 
Examinations. Besides, tests can be tailored to the stakeholders' needs because the 
hiring company may suggest what it needs to be observed, the taker only has to perform 
tasks which are relevant to the job (or situation), the delivery institution only devotes 
those resources that are required (not the whole set of raters and test administrators). 



Advice to other institutions and test deliverers 



Language testing must be flexible and adaptable to its needs. Relying on fixed testing 
structures may be a major drawback when designing online language tests. Reducing 
costs and developing multimodal platfonns is a must in language testing. Good 
academic developments offer great opportunities to benefit all the stakeholders 
especially the designers and central administrators. On the other hand, these platforms 
offer tailored responses to the most specific demands. Designing platforms specifically 
aimed to certain tests may be valuable and certainly a solution for large organizations 
such as ETS or the Instituto Cervantes (to mention just two) but low stakes tests and 
platfonns certainly are less costly and still produce significant revenues. The advantage 
of these platforms is their flexibility and that fact that private companies do not need 
highly secure platforms (like ETS or the typical University Entrance Examination in 
Spain), but rather efficient and low cost examination systems. Additionally, the 






flexibility is adding value, because these platfonns can also work as a part of language 
courses platforms and, in this way, reduce the costs of the academic or commercial 
courses through the revenues produced by the platforms' commercial use. 



Note 



I would like to express my gratitude to Christine Engelen (Linguapolis, Antwerpen 
University) for pushing me (literally) to write this paper and giving me her support and 
ideas and also to the Ministry of Education of Spain for supporting the development and 
implementation of the testing platform hereby mentioned under the PAER project 
(HUM2007-66479-C02-0 1/FILO). 
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